{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "23b97635",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "# 线性代数\n",
    "\n",
    "标量由只有一个元素的张量表示"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "8428fe2d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:49:33.083537Z",
     "iopub.status.busy": "2022-12-07T16:49:33.076823Z",
     "iopub.status.idle": "2022-12-07T16:49:34.322234Z",
     "shell.execute_reply": "2022-12-07T16:49:34.321388Z"
    },
    "origin_pos": 2,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor(5.), tensor(6.), tensor(1.5000), tensor(9.))"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import torch\n",
    "\n",
    "x = torch.tensor(3.0)\n",
    "y = torch.tensor(2.0)\n",
    "\n",
    "x + y, x * y, x / y, x**y"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d87f02f7",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "向量可以被视为标量值组成的列表"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "c993f620",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:49:34.326689Z",
     "iopub.status.busy": "2022-12-07T16:49:34.325804Z",
     "iopub.status.idle": "2022-12-07T16:49:34.331693Z",
     "shell.execute_reply": "2022-12-07T16:49:34.330962Z"
    },
    "origin_pos": 7,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([0, 1, 2, 3])"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = torch.arange(4)\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a187de5",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "通过张量的索引来访问任一元素"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "e6fd7ba7",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:49:34.335049Z",
     "iopub.status.busy": "2022-12-07T16:49:34.334579Z",
     "iopub.status.idle": "2022-12-07T16:49:34.340161Z",
     "shell.execute_reply": "2022-12-07T16:49:34.339403Z"
    },
    "origin_pos": 12,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(3)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[3]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ddbe762e",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "访问张量的长度"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "8f60ef45",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:49:34.343613Z",
     "iopub.status.busy": "2022-12-07T16:49:34.343159Z",
     "iopub.status.idle": "2022-12-07T16:49:34.348385Z",
     "shell.execute_reply": "2022-12-07T16:49:34.347571Z"
    },
    "origin_pos": 17,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d2160550",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "只有一个轴的张量，形状只有一个元素"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "68936fe1",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:49:34.353277Z",
     "iopub.status.busy": "2022-12-07T16:49:34.352830Z",
     "iopub.status.idle": "2022-12-07T16:49:34.357906Z",
     "shell.execute_reply": "2022-12-07T16:49:34.357134Z"
    },
    "origin_pos": 22,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([4])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e77b5b48",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "通过指定两个分量$m$和$n$来创建一个形状为$m \\times n$的矩阵"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "3b0271e8",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:49:34.363949Z",
     "iopub.status.busy": "2022-12-07T16:49:34.363344Z",
     "iopub.status.idle": "2022-12-07T16:49:34.369465Z",
     "shell.execute_reply": "2022-12-07T16:49:34.368686Z"
    },
    "origin_pos": 27,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0,  1,  2,  3],\n",
       "        [ 4,  5,  6,  7],\n",
       "        [ 8,  9, 10, 11],\n",
       "        [12, 13, 14, 15],\n",
       "        [16, 17, 18, 19]])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = torch.arange(20).reshape(5, 4)\n",
    "A"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7566710a",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "矩阵的转置"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "1a1619e7",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:49:34.374581Z",
     "iopub.status.busy": "2022-12-07T16:49:34.373999Z",
     "iopub.status.idle": "2022-12-07T16:49:34.379696Z",
     "shell.execute_reply": "2022-12-07T16:49:34.378939Z"
    },
    "origin_pos": 32,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0,  4,  8, 12, 16],\n",
       "        [ 1,  5,  9, 13, 17],\n",
       "        [ 2,  6, 10, 14, 18],\n",
       "        [ 3,  7, 11, 15, 19]])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A.T"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8863d576",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "*对称矩阵*（symmetric matrix）$\\mathbf{A}$等于其转置：$\\mathbf{A} = \\mathbf{A}^\\top$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "49554f58",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:49:34.385383Z",
     "iopub.status.busy": "2022-12-07T16:49:34.384746Z",
     "iopub.status.idle": "2022-12-07T16:49:34.391063Z",
     "shell.execute_reply": "2022-12-07T16:49:34.390290Z"
    },
    "origin_pos": 37,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1, 2, 3],\n",
       "        [2, 0, 4],\n",
       "        [3, 4, 5]])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "B = torch.tensor([[1, 2, 3], [2, 0, 4], [3, 4, 5]])\n",
    "B"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "29a3fe08",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:49:34.396525Z",
     "iopub.status.busy": "2022-12-07T16:49:34.395968Z",
     "iopub.status.idle": "2022-12-07T16:49:34.401470Z",
     "shell.execute_reply": "2022-12-07T16:49:34.400734Z"
    },
    "origin_pos": 42,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[True, True, True],\n",
       "        [True, True, True],\n",
       "        [True, True, True]])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "B == B.T"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a02a4ad5",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "就像向量是标量的推广，矩阵是向量的推广一样，我们可以构建具有更多轴的数据结构"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "6026dc3e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:49:34.406847Z",
     "iopub.status.busy": "2022-12-07T16:49:34.406292Z",
     "iopub.status.idle": "2022-12-07T16:49:34.412252Z",
     "shell.execute_reply": "2022-12-07T16:49:34.411463Z"
    },
    "origin_pos": 47,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[[ 0,  1,  2,  3],\n",
       "         [ 4,  5,  6,  7],\n",
       "         [ 8,  9, 10, 11]],\n",
       "\n",
       "        [[12, 13, 14, 15],\n",
       "         [16, 17, 18, 19],\n",
       "         [20, 21, 22, 23]]])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = torch.arange(24).reshape(2, 3, 4)\n",
    "X"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8bde21c4",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "给定具有相同形状的任意两个张量，任何按元素二元运算的结果都将是相同形状的张量"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "daf13e51",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:49:34.416832Z",
     "iopub.status.busy": "2022-12-07T16:49:34.416295Z",
     "iopub.status.idle": "2022-12-07T16:49:34.423678Z",
     "shell.execute_reply": "2022-12-07T16:49:34.422913Z"
    },
    "origin_pos": 52,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([[ 0.,  1.,  2.,  3.],\n",
       "         [ 4.,  5.,  6.,  7.],\n",
       "         [ 8.,  9., 10., 11.],\n",
       "         [12., 13., 14., 15.],\n",
       "         [16., 17., 18., 19.]]),\n",
       " tensor([[ 0.,  2.,  4.,  6.],\n",
       "         [ 8., 10., 12., 14.],\n",
       "         [16., 18., 20., 22.],\n",
       "         [24., 26., 28., 30.],\n",
       "         [32., 34., 36., 38.]]))"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = torch.arange(20, dtype=torch.float32).reshape(5, 4)\n",
    "B = A.clone()\n",
    "A, A + B"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb840664",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "两个矩阵的按元素乘法称为*Hadamard积*（Hadamard product）（数学符号$\\odot$）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "3fcf4677",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:49:34.427273Z",
     "iopub.status.busy": "2022-12-07T16:49:34.426684Z",
     "iopub.status.idle": "2022-12-07T16:49:34.432875Z",
     "shell.execute_reply": "2022-12-07T16:49:34.432097Z"
    },
    "origin_pos": 57,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[  0.,   1.,   4.,   9.],\n",
       "        [ 16.,  25.,  36.,  49.],\n",
       "        [ 64.,  81., 100., 121.],\n",
       "        [144., 169., 196., 225.],\n",
       "        [256., 289., 324., 361.]])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A * B"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "f4642bf4",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:49:34.436554Z",
     "iopub.status.busy": "2022-12-07T16:49:34.435975Z",
     "iopub.status.idle": "2022-12-07T16:49:34.442297Z",
     "shell.execute_reply": "2022-12-07T16:49:34.441534Z"
    },
    "origin_pos": 62,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([[[ 2,  3,  4,  5],\n",
       "          [ 6,  7,  8,  9],\n",
       "          [10, 11, 12, 13]],\n",
       " \n",
       "         [[14, 15, 16, 17],\n",
       "          [18, 19, 20, 21],\n",
       "          [22, 23, 24, 25]]]),\n",
       " torch.Size([2, 3, 4]))"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = 2\n",
    "X = torch.arange(24).reshape(2, 3, 4)\n",
    "a + X, (a * X).shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e05e71f",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "计算其元素的和"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "0f1dba19",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:49:34.445873Z",
     "iopub.status.busy": "2022-12-07T16:49:34.445289Z",
     "iopub.status.idle": "2022-12-07T16:49:34.451676Z",
     "shell.execute_reply": "2022-12-07T16:49:34.450932Z"
    },
    "origin_pos": 67,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([0., 1., 2., 3.]), tensor(6.))"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = torch.arange(4, dtype=torch.float32)\n",
    "x, x.sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7bec2c44",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "表示任意形状张量的元素和"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "8cdcb2ff",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:49:34.455101Z",
     "iopub.status.busy": "2022-12-07T16:49:34.454535Z",
     "iopub.status.idle": "2022-12-07T16:49:34.460146Z",
     "shell.execute_reply": "2022-12-07T16:49:34.459392Z"
    },
    "origin_pos": 72,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(torch.Size([5, 4]), tensor(190.))"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A.shape, A.sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26464e38",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "指定张量沿哪一个轴来通过求和降低维度"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "6e8f71da",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:49:34.463608Z",
     "iopub.status.busy": "2022-12-07T16:49:34.463028Z",
     "iopub.status.idle": "2022-12-07T16:49:34.468918Z",
     "shell.execute_reply": "2022-12-07T16:49:34.468151Z"
    },
    "origin_pos": 77,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([40., 45., 50., 55.]), torch.Size([4]))"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A_sum_axis0 = A.sum(axis=0)\n",
    "A_sum_axis0, A_sum_axis0.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "e148213f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:49:34.472411Z",
     "iopub.status.busy": "2022-12-07T16:49:34.471822Z",
     "iopub.status.idle": "2022-12-07T16:49:34.477842Z",
     "shell.execute_reply": "2022-12-07T16:49:34.477046Z"
    },
    "origin_pos": 82,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([ 6., 22., 38., 54., 70.]), torch.Size([5]))"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A_sum_axis1 = A.sum(axis=1)\n",
    "A_sum_axis1, A_sum_axis1.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "572a733a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:49:34.481047Z",
     "iopub.status.busy": "2022-12-07T16:49:34.480756Z",
     "iopub.status.idle": "2022-12-07T16:49:34.486930Z",
     "shell.execute_reply": "2022-12-07T16:49:34.486142Z"
    },
    "origin_pos": 87,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(190.)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A.sum(axis=[0, 1])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7142e3c4",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "一个与求和相关的量是*平均值*（mean或average）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "72e50902",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:49:34.490604Z",
     "iopub.status.busy": "2022-12-07T16:49:34.489990Z",
     "iopub.status.idle": "2022-12-07T16:49:34.496680Z",
     "shell.execute_reply": "2022-12-07T16:49:34.495883Z"
    },
    "origin_pos": 92,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor(9.5000), tensor(9.5000))"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A.mean(), A.sum() / A.numel()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "0d826cfd",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:49:34.500565Z",
     "iopub.status.busy": "2022-12-07T16:49:34.499849Z",
     "iopub.status.idle": "2022-12-07T16:49:34.506752Z",
     "shell.execute_reply": "2022-12-07T16:49:34.505915Z"
    },
    "origin_pos": 97,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([ 8.,  9., 10., 11.]), tensor([ 8.,  9., 10., 11.]))"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A.mean(axis=0), A.sum(axis=0) / A.shape[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b80470b0",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "计算总和或均值时保持轴数不变"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "581b7536",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:49:34.510528Z",
     "iopub.status.busy": "2022-12-07T16:49:34.509903Z",
     "iopub.status.idle": "2022-12-07T16:49:34.516165Z",
     "shell.execute_reply": "2022-12-07T16:49:34.515399Z"
    },
    "origin_pos": 102,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 6.],\n",
       "        [22.],\n",
       "        [38.],\n",
       "        [54.],\n",
       "        [70.]])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sum_A = A.sum(axis=1, keepdims=True)\n",
    "sum_A"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27bd46ab",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "通过广播将`A`除以`sum_A`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "97fa6ce7",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:49:34.519709Z",
     "iopub.status.busy": "2022-12-07T16:49:34.519131Z",
     "iopub.status.idle": "2022-12-07T16:49:34.524883Z",
     "shell.execute_reply": "2022-12-07T16:49:34.524124Z"
    },
    "origin_pos": 107,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.0000, 0.1667, 0.3333, 0.5000],\n",
       "        [0.1818, 0.2273, 0.2727, 0.3182],\n",
       "        [0.2105, 0.2368, 0.2632, 0.2895],\n",
       "        [0.2222, 0.2407, 0.2593, 0.2778],\n",
       "        [0.2286, 0.2429, 0.2571, 0.2714]])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A / sum_A"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3bf01659",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "某个轴计算`A`元素的累积总和"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "ab93a7d7",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:49:34.528341Z",
     "iopub.status.busy": "2022-12-07T16:49:34.527777Z",
     "iopub.status.idle": "2022-12-07T16:49:34.533740Z",
     "shell.execute_reply": "2022-12-07T16:49:34.532986Z"
    },
    "origin_pos": 112,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.,  1.,  2.,  3.],\n",
       "        [ 4.,  6.,  8., 10.],\n",
       "        [12., 15., 18., 21.],\n",
       "        [24., 28., 32., 36.],\n",
       "        [40., 45., 50., 55.]])"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A.cumsum(axis=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1ca9d8c7",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "点积是相同位置的按元素乘积的和"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "f2290d13",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:49:34.537131Z",
     "iopub.status.busy": "2022-12-07T16:49:34.536609Z",
     "iopub.status.idle": "2022-12-07T16:49:34.543465Z",
     "shell.execute_reply": "2022-12-07T16:49:34.542632Z"
    },
    "origin_pos": 117,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([0., 1., 2., 3.]), tensor([1., 1., 1., 1.]), tensor(6.))"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = torch.ones(4, dtype = torch.float32)\n",
    "x, y, torch.dot(x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34167720",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "我们可以通过执行按元素乘法，然后进行求和来表示两个向量的点积"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "f32e6b06",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:49:34.547022Z",
     "iopub.status.busy": "2022-12-07T16:49:34.546435Z",
     "iopub.status.idle": "2022-12-07T16:49:34.552564Z",
     "shell.execute_reply": "2022-12-07T16:49:34.551736Z"
    },
    "origin_pos": 122,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(6.)"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.sum(x * y)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2349b662",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "矩阵向量积$\\mathbf{A}\\mathbf{x}$是一个长度为$m$的列向量，\n",
    "其第$i$个元素是点积$\\mathbf{a}^\\top_i \\mathbf{x}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "0b2aed0d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:49:34.556331Z",
     "iopub.status.busy": "2022-12-07T16:49:34.555742Z",
     "iopub.status.idle": "2022-12-07T16:49:34.561968Z",
     "shell.execute_reply": "2022-12-07T16:49:34.561191Z"
    },
    "origin_pos": 130,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(torch.Size([5, 4]), torch.Size([4]), tensor([ 14.,  38.,  62.,  86., 110.]))"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A.shape, x.shape, torch.mv(A, x)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e94cd3cb",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "我们可以将矩阵-矩阵乘法$\\mathbf{AB}$看作简单地执行$m$次矩阵-向量积，并将结果拼接在一起，形成一个$n \\times m$矩阵"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "763fbcb9",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:49:34.565474Z",
     "iopub.status.busy": "2022-12-07T16:49:34.564932Z",
     "iopub.status.idle": "2022-12-07T16:49:34.571397Z",
     "shell.execute_reply": "2022-12-07T16:49:34.570560Z"
    },
    "origin_pos": 135,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 6.,  6.,  6.],\n",
       "        [22., 22., 22.],\n",
       "        [38., 38., 38.],\n",
       "        [54., 54., 54.],\n",
       "        [70., 70., 70.]])"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "B = torch.ones(4, 3)\n",
    "torch.mm(A, B)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12d874c6",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "$L_2$*范数*是向量元素平方和的平方根：\n",
    "$$\\|\\mathbf{x}\\|_2 = \\sqrt{\\sum_{i=1}^n x_i^2}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "0d07478e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:49:34.575336Z",
     "iopub.status.busy": "2022-12-07T16:49:34.574783Z",
     "iopub.status.idle": "2022-12-07T16:49:34.580907Z",
     "shell.execute_reply": "2022-12-07T16:49:34.580098Z"
    },
    "origin_pos": 140,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(5.)"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "u = torch.tensor([3.0, -4.0])\n",
    "torch.norm(u)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8f6e885a",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "$L_1$范数，它表示为向量元素的绝对值之和：\n",
    "$$\\|\\mathbf{x}\\|_1 = \\sum_{i=1}^n \\left|x_i \\right|$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "3230c3a4",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:49:34.584423Z",
     "iopub.status.busy": "2022-12-07T16:49:34.583891Z",
     "iopub.status.idle": "2022-12-07T16:49:34.589927Z",
     "shell.execute_reply": "2022-12-07T16:49:34.589143Z"
    },
    "origin_pos": 145,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(7.)"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.abs(u).sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef8adc42",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "矩阵\n",
    "的*Frobenius范数*（Frobenius norm）是矩阵元素平方和的平方根：\n",
    "$$\\|\\mathbf{X}\\|_F = \\sqrt{\\sum_{i=1}^m \\sum_{j=1}^n x_{ij}^2}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "d70feaaa",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:49:34.593416Z",
     "iopub.status.busy": "2022-12-07T16:49:34.592912Z",
     "iopub.status.idle": "2022-12-07T16:49:34.599045Z",
     "shell.execute_reply": "2022-12-07T16:49:34.598210Z"
    },
    "origin_pos": 150,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(6.)"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.norm(torch.ones((4, 9)))"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "language_info": {
   "name": "python"
  },
  "rise": {
   "autolaunch": true,
   "enable_chalkboard": true,
   "overlay": "<div class='my-top-right'><img height=80px src='http://d2l.ai/_static/logo-with-text.png'/></div><div class='my-top-left'></div>",
   "scroll": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}